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Abstract 


Learning essential information literacy skills through the use of mobile phones is an innovative m- 
learning pilot project that was collaboratively undertaken in a Canadian university college over the 
course of two academic terms by faculty and the library staff. The research pilot project involved 
ninety one undergraduate students in five different classes majoring in psychology, social work, 
education or social development studies in an attempt to determine the effectiveness of using 
mobile technology to enhance students’ information literacy skills and learning experiences. Pre 
and post-test measures, and survey questionnaires generated quantitative and qualitative data that 
was analyzed to determine the degree of changes in frequency of mobile device information literacy 
access and fluency in digital literacy skills. The article highlights the Mobile Information Literacy 
innovation and includes the development and design of the mobile lessons, interactive exercises, 
and its applications. The study’s main results and conclusions are also discussed. Additionally, the 
successes and challenges of the pilot to support anytime, anywhere student mobile information 
literacy eLearning training that engages mobile learners and enhances their learning experience are 
identified and critically reflected upon to improve the innovation for stage two of the project. 

Keywords: Mobile learning; information literacy; e-Leaming; digital literacy; post-secondary 
education. 
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Introduction 

University students frequently use academic material both on and off campus, there is demonstrated 
need to provide easily accessible tools to assist them (Baggaley, 2004; Brown, 2004; Perry, 2000). 
Academic literature has emphasized the value of teaching information literacy skills, clearly linked 
with academic and critical thinking skills, as part of a comprehensive university education (Kim & 
Shumaker, 2015; MacPherson, 2004; Tumbleson & Burke, 2013). Information literacy is 
commonly defined as the ability to locate, to access, evaluate, and use information that cuts across 
all disciplines, all learning environments, and all levels of education (Association of College & 
Research Libraries' Information Literacy Competency Standards for Higher Education, 2000; 
Saunders, 2012). Despite the aspiration towards pedagogical innovation in higher education, the 
inclusion of information literacy in undergraduate education often remains an objective rather than 
a fully realized ideal/accomplishment 

This project supported the development and administration of the mobile information literacy (MIL) 
tool, being a web search application for use by undergraduate arts and humanities students at 
Renison University College, affiliated with the University of Waterloo, Waterloo, Ontario, Canada. 
Participants had the opportunity to utilize this tool and reported on their level of satisfaction with 
the tool. Project objectives were to develop best strategies, from a user perspective, for delivering 
and accessing information that enhances student information literacy skills through mobile 
technology. With the MIL tool, the research team hypothesized that students would improve their 
literacy skills and would increase their access, retrieval and evaluation skills, to ascertain and 
understand reliable and credible academic information. The research team also felt that other school 
communities would benefit from the MIL tool to assist them to complete academic assignments 
and research projects. Our research aims to contribute to the understanding of the innovative 
practice for mobile technology academic learning (m-leaming). 

The pilot study was a mixed-method (quantitative and qualitative) non-experimental approach that 
included both pre- and post- digital literacy tests and student questionnaires. All study participants 
completed a digital literacy pre-test and a digital literacy post-test. Undergraduates in three 
participant groups completed thirteen mobile information literacy lessons (online) before 
completing the post-test and questionnaire. Students in the comparison group received a one hour 
research skills library workshop and completed the pre and post digital literacy test. The 
comparison group participants did not participate in the thirteen online literacy lessons. 

Literature Review 

M-leaming involves the use of mobile devices to deliver electronic learning materials with built-in 
learning strategies to allow access to knowledge from anywhere and at any time (Ally, 2004). M- 
leaming or “education on the go” utilizing mobile devices such as mobile phones and tablets, 
expands the boundaries of anytime, anywhere learning and will play an important role in the future 
of learning (Keegan, 2002; Wu et al., 2012). As this type of learning is an emerging field, the full 
potential of m-leaming is still untapped and best-practice guidelines for m-leaming are still 
unknown. Although using mobile technology for information literacy training is limited, there are 
a few programs in universities and colleges in the United States, England and Australia that include 
infusing information literacy and technology into the educational experience in for-credit courses 
and a certificate provided upon graduation for completion of the lessons (DaCosta, 2010; Kraemer 
et al., 2007; Salisbury& Ellis, 2003; Wamken, 2004). 

Research on m- learning is a recent development and there has been limited research conducted in 
this area (Attewell, 2005; British Educational Communications Technology Agency, 2004; Keegan, 
2002; Savill-Smith & Kent, 2003). A study conducted by DaCosta (2010) on the integration of 
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information literacy skills in selected post-secondary higher education curricula in England and the 
United States found limited integration of information literacy in the curriculum across a variety of 
academic disciplines, including arts, education, sciences and engineering. Although information 
literacy skills were deemed to be important tools by faculty and librarians for students, 
opportunities for integration were limited. There appears to be no consensus among faculty on 
when students should learn the skills or if they need to be taught information literacy in the 
curriculum (DaCosta, 2010). This highlights the gap between understanding the level of importance 
of the information literacy skills and embedding them into the curriculum. Another identified 
challenge is the misperception by some faculty that computer literacy equals information literacy 
(Salisbury & Ellis, 2003). Osmosis does not work for the development of such skills, but rather 
pedagogical collaborations between faculty and librarians can be encouraged and established to 
assist in incorporating information literacy into higher education curriculums. 

Typically, information literacy competencies are highly valued by faculty in various disciplines at 
colleges and universities, however little has been written on information literacy outside of the 
library literature. The transformation of information literacy from a library-centered issue to a 
mainstream educational issue is just beginning (DaCosta, 2010; McGuinness, 2006; Owusu-Ansah, 
2004). Often, there is limited interaction between faculty and librarians and the seeming 
unwillingness of academic faculty to partner with librarians may have less to do with a lack of 
respect for the position, and more to do with a lack of understanding of how librarians can 
contribute to and support their instruction (Saunders, 2012). The pilot study described in this paper 
aimed to enhance the collaborative efforts between these two roles in order to allow them to share 
responsibility for teaching students information literacy skills in an innovative way. 

More research, from a user perspective, is needed to discover the best strategies for maximizing m- 
leaming, including discovering what is the best mobile device for accessing digital information 
tools such as the MIL, and what form the content needs to take to contribute to effective m-learning 
to engage mobile learners. The passion to help student learners improve their information skills 
using mobile devices shaped the research framework for this project. Overall, the research on the 
educational use of mobile devices is in the early stages and limited research includes case studies 
of different implementations. Cross-faculty and library collaborations are still needed. 

Outcomes of this project will have several meaningful and significant contributions to the emerging 
knowledge in the field of m-learning. To be successful and independent learners for life, students 
must graduate with the ability to successfully navigate electronic environments. Understanding and 
using both the information and technology related to their fields of study is deemed critical. With 
emergence of new technology, ways to develop information and digital literacy skills in the 
curriculum that interact with mobile technology offers exciting possibilities (Sandars, 2012; 
Saunders, 2012). 

We ask the question, “Is m-learning the next evolutionary step of e-learning?” To fill the gap in the 
research, this innovative project was designed and implemented to support and enhance m-learning 
pedagogy at Renison University College. Students enrolled in both the Bachelor of Social Work 
and the Bachelor of Social Development Studies degree programs are required to write numerous 
academic papers over the course of their programs. Many students struggle with information 
literacy in searching for appropriate information and evaluating the validity of sources. It is 
especially critical that students understand how to conduct research and be self-reliant in the 
electronic information environment at a time when there is less need to consult with a librarian or 
to access a library. 
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It is anticipated that m-learning will grow quickly in the next few years. With the development of 
a variety of mobile devices that are more powerful, portable and have increased Wi-Fi access, this 
research will serve as a foundation for developing and promoting segments of mobile learning 
among students, the community and beyond. There is an apparent gap between the information 
literacy skills that faculty want their students to have and those that they actively support and 
develop. It is a gap that faculty and librarians from various faculties are best placed to fill as 
collaborators and bridge builders. This project begins this collaborative, bridge-building process. 

Methodology and methods 

Our study used a mixed-method (quantitative and qualitative) non-experimental approach, 
including both pre- and post- literacy tests and student questionnaires. This project and the survey 
instruments were approved by the Research Ethics Board at the University of Waterloo. Ninety- 
one university undergraduate students participated in the project during the fall 2014 and winter 
2015 school semesters as part of their course activities from five Social Development Studies (SDS) 
and Social Work classes at Renison University College. University participants ranged in year of 
study from their second year to their fourth year. Undergraduates in four of the five participant 
groups completed a pre-test, thirteen mobile information literacy lessons (online) before 
completing the post-test and questionnaire. These students accessed their personal smart phones 
and tablets to complete the exercises in their spare time (e.g. riding a bus). Students in the 
comparison group (n=32) received a one hour research skills library workshop and completed the 
pre and post digital literacy test, but they did not participate in the thirteen online literacy lessons. 

A short training session on the use of the MIL tool was provided by the librarian to students before 
completing the on-line lessons. At the start of the session, participants completed paper and pencil 
pre-tests to determine a baseline understanding of information literacy. On-line student participants 
completed pre-test/post-test surveys through a web link accessed using their course newsfeed in 
the university’s course management learning system. Survey questions explored participants’ 
knowledge about accessing data, including the university’s library database system, Primo. 
Administration of pre and post-test surveys occurred during the first week of classes and in the 
final week of the term respectively. Additionally, students provided an assessment of their use of 
the MIL tool at the end of the semester. All study participants received a 1% bonus mark in their 
course at the end of the completion of the study. 

Both quantitative and qualitative data was collected and coded. Statistical analysis of the completed 
surveys and questionnaires was done using Survey Monkey's Analyze tool and by reviewing the 
raw data through the program Wordpress (https://wordpress.org). Opened ended questions were 
coded and thematically analyzed while usage of the MIL web app tool was explored through 
Google Analytics. The data was analyzed for program improvement, MIL tool enhancement and 
expansion, and as basic research in the emerging field of information literacy instruction. 

Findings 

Data collected through a Survey Monkey link, an online cloud-based survey tool, retained 
participant demographic information and survey results. Almost 60% of students were in a post¬ 
degree Bachelor of Social Work program, while 25% of students were in an undergraduate SDS 
program and the remaining students (about 15%) identified their programs as other arts faculty or 
humanities programs (psychology, sociology, Lrench or fine arts). The comparison group 
demographics closely matched with other participant groups and consisted of thirty two Bachelor 
of Social Work students. Most study participants were female (90%) and 77% of the participants 
were between the ages of eighteen and twenty-five, while 3.5% of participants indicated they were 
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over fifty years of age. Prior to participating in this MIL pilot study, almost 87% of students had 
not received any type of literacy skills training. 

Close to ninety-eight percent of participants owned a smartphone and 58% of these participants 
were Apple iPhone users. This group reported using their phone and other mobile devices (i.e. 
tablets) on a daily basis. Only twenty-eight percent of this group used a mobile device to search for 
academic related information despite daily usage, while thirty-six percent of them made phone calls 
with their device. Texting was their main use (86%), while 35% browsed the internet and less than 
10% of students played games on their smartphones. 

Students who accessed academic information or conducted research on their phones significantly 
preferred using a Google search engine (69%)over other search tools, such as the University of 
Waterloo’s research and discovery tool, Primo (29%), other research databases, including Google 
Scholar (30%), or RefWorks (3%), a web-based citation and bibliography tool. 

Variations in accessing the mobile lessons by study participants were noted (see Table 1) as some 
lessons proved to be more frequently accessed than others. 

Table 1. Access to the mobile lessons by study participants 


Lessons 

Locate: Lesson 1: An Introduction to Primo Central 

Locate: Lesson 2: The Basics: How to Search 

Locate: Lesson 3: Finding Articles 

Locate: Lesson 4: Finding Peer Reviewed Journals 

Evaluate: Lesson 5: Evaluating Information Sources 

Evaluate: Lesson 6: Peer Review Process 

Evaluate: Lesson 7: Popular Vs. Scholarly Resources 

Use: Lesson 8: Using the Web for Resources 

Use: Lesson 9: RefWorks 

Use: Lesson 10: When to Cite Your Articles 

Use: Lesson 11: Types of Written Articles 

Use: Lesson 12: How to Approach Assignments 

Use: Lesson 13: What is a Paragraph? 


Response Percent 

60.2% 

44% 

78.3% (3) 
82.3% (1) 
73.3% (4) 
60% 

73.2% (5) 
67.9% 
61.3% 
79.3% (2) 
64.1% 
68 . 2 % 
56.8% 


Seventy-nine participants completed the pre and post-tests, the MIL questionnaire, and all thirteen 
information lessons. A majority of those participants (77.2%) gained or maintained their MIL 
knowledge as evidenced by the test results. Of those, 50.63% improved their information literacy 
knowledge while 26.58% maintained their knowledge from beginning to end of the semester. There 
was a decrease in test scores noted in 23% of participants. 
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Study Findings: Information Literacy Knowledge 

Several key findings emerged during the review of the information literacy knowledge pre and 
post-test scores for participant and comparison group scores (see Table 2). These included: 1) three 
of the four participant group score averages (participant groups A,B, and D) increased substantially 
(about 50%), 2) Group B, who had the least number of participants (n=4), varied significantly in 
final scores in relation to all other groups and increases in knowledge were not reported, 3) many 
of the online participants (75%) had significant improvements in their post test scores, 4) over one- 
quarter (28%) of the study participant’s information literacy knowledge decreased from beginning 
to the end of the semester while about half that number (12.5%) of comparison participants 
information literacy knowledge declined, and 5) comparison group participant post-test scores were 
surprisingly strong and actually surpassed the post test scores of the participant students that 
completed the MIL lessons via smartphones. 

Table 2. Information Literacy Knowledge Pre-Post Test Score Results 



Number of 
participants 

Increase 

N % 

Decrease 

N % 

Maintain 

N % 

Total 

% 

TOTAL participants 99 

On-campus participants 

A. SDS-01 

27 

16 

59.3% 

7 

25.9% 

4 

14.8% 

100% 

B. SDS-02 

4 

0 

0% 

3 

75% 

1 

25% 

100% 

C. Social Work 

16 

8 

50% 

4 

25% 

4 

25% 

100% 

Online participants 

D. Social Work 

online 

20 

13 

65% 

5 

25% 

2 

10% 

100% 

Participants 

(A+B+C+D) 

67 

37 

55.2% 

19 

28.4% 

11 

16.4% 

100% 

Comparison group 









E. Comparison 
Group (SW) 

32 

16 

50% 

4 

12.5% 

12 

37.5% 

100% 


Notable enhancements (55%) in information literacy knowledge occurred for three of the four 
participant groups from the beginning to the end of the semester, whether they completed the 
mobile lessons online or through their smartphones (see Table 3). For participant group A (n=27), 
a class in SDS, almost 60% (16 of 27) of the students demonstrated an increase in information 
literacy knowledge from the start of classes to the end of the semester. Almost 38% maintained the 
information literacy knowledge throughout the semester. These students were required to write 
research paper proposals, which were then used as the starting point for their course research papers. 
Perhaps group A students were able to apply what they learned from the MIL tool directly to their 
course assignments. 

The smallest participant group B (n=4) varied significantly from the other groups. This small group 
did not improve their information literacy scores by the end of the semester (0%), rather the scores 
decreased for three of the four students while the fourth student retained the information literacy 
knowledge from the beginning to the end of the semester. As voluntary participation in this project 
was offered to students and only four volunteered from this class, perhaps the results may not be 
representative due to small numbers. 

Seventy-five percent (12 of 16 students) in the social work Group C (n= 16) participant group 
retained or increased their information literacy knowledge over the course of the semester. Online 
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learners, Group D (n=20) demonstrated the greatest increases (65%) in information literacy of the 
study participants. Seventy-five percent of those online students who increased their post test scores 
recorded gains of thirty to forty percent. Could this be a reflection of a student’s familiarity and 
comfort with online learning where they easily gravitated to the online modules? 

Scores between the pre and post-tests decreased for some students in all participant groups 
indicating a loss of information literacy knowledge. The loss of information literacy knowledge 
was greater for those students in the online class (about 25%) than the participants in the 
comparison group (12.5%). The decline could reflect the negative impact on study participant 
memory recall arising from the lapse in time between the completion of lessons early in the 
semester and the post-test many weeks later. 

Comparison group E members that received the in-class information literacy tutorial demonstrated 
significant gains during the semester. Twelve comparison group students (40%) maintained their 
information literacy knowledge at the same level, as compared to twenty (65%) of the online Social 
Work class. It is interesting to note that over eighty-seven percent of the students who received 
their information literacy lesson in person retained or increased their information literacy 
knowledge as compared to forty-eight (71.6%) of the students who received the information via 
their smartphones. Not all students in the participant groups completed all the thirteen lessons 
which may explain the decline in their follow up test scores. The research team also recognizes this 
is a pilot study and future development, enhancement and refinement of the MIL tool, pre and post 
tests and the questionnaire are necessary which may impact outcomes and may address the variance 
in scores. 

Table 3. Information Literacy knowledge Pre-Post Test 



The authors of this paper present the study findings in Table 3 as preliminary results and a reflection 
on lessons learned from our pilot study. These results will inform Stage 2 of the MIL project e.g. 
testing the revised MIL tool, revised pre/post-test surveys and questionnaires, and a larger sample 
size. 

Description of the MIL Tool 
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The MIL web app contains 13 lessons, focusing on how to locate, evaluate and use information 
(Figure 1) and is accessible at URL: http://beam.to/renmil. The learning contents are designed for 
use with the University of Waterloo’s library resources and services. The lessons contain multiple 
instructional videos about topics such as Boolean operators, database functions, writing search 
strategies using keywords, and bibliographic management tools such as Refworks. 
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Mobile Information Literacy | Modul... 


Mobile Information Literacy 

Modules & Quizzes 


Welcome to the Digital Literacy Hub! 

July 7,2014 /Welcome 


Locate 

Evaluate 

Use 



Figure 1. Web App 

Many information literacy studies indicate that interactivity and assessment can help to reinforce 
concepts learned (Grassian & Kaplowitz, 2001; Yarmey, 2011). For these reasons, each MIL lesson 
has interactive exercises providing instant feedback including True/False, Multiple Choice, Text 
Impute, and Drag and Drop as demonstrated in Figure 2. 
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True/False Multiple Choice Text input 



Quiz: The Basics: How to Search 

I v * 

■ Answered ■ Review 

Rente* queokin QtAewnmary 


1 *ou are currently In the database engine PSrCHirlo and you want to look for the EXACT 
phrase: testing effect 

How should you type the prwase Into the search bar? 

7. »ou decide that you want go look lor a different EXACT phrase media impact 
How should you type the pnrase into the search bar? 








Drag and Drop 


Figure 2. Different types of interactive exercises 


The web app also provides two types of assessment tools to evaluate students’ progress as shown 
in Figure 3. The formative assessment allows students to complete one question and then view their 
results before moving on to subsequent questions. The summative evaluation permits student to 
view their results at the end of the lesson exercise and to compare their results with peers. 

The web app also tracks the amount of time that students spend completing the exercises. The web 
app supports knowledge construction, dissemination and collective intelligence, by encouraging 
students to post their search tips at the mobile friendly site. 


Formative Evaluation 


Summative Evaluation 
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What is the second step of the Peer-Reviewed Process? 
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O Write up a draft article and submit it to a journal. 
O Thinking of a few research ideas. 

Incorrect 

Give it one more try. 

2. Question 

What is the third step of the Peer-Reviewed Process? 



Quiz: The Basics: How to Search 

Results 

0 of 4 questions answered correctly 
Your time: 00:00:53 


You have reached 0 of 4 points, (0%) 

Average score 22.22% 

Your score 0% 


Your result has been entered into leaderboard 

Name: Name E-Mail: E-Mail 

Send 


Restart quiz 



Figure 3. Summative and formative response 

The MIL web app runs on WordPress (https://wordpress.org), an open source content management 
system based on MYSQL and PHP. It generates content on the fly without any html programming 
involved. MIL utilizes a mobile plugin to auto-detect the mobile browser and load the mobile 
version of the MIL site. For assessment, a plug-in program called Wp-Pro-Quiz generates different 
types of quizzes and the web app also tracks users’ scores and generates statistical reports. Google 
Analytics was used to analyze student use and engagement, especially time spent on the MIL site 
and detects the content students interacted with the most. 

Discussion 

MIL training for students, especially mobile learners will enhance mobility and flexibility in 
learning, as well as enable students to be “spontaneous, personal, informal, contextual, portable, 
ubiquitous, and pervasive” (Kukulska-Hume, 2005, p.5). Most participants in the pilot study 
expressed a positive experience using the mobile web app technology to leam information literacy 
skills, and appreciated the flexibility of “anytime” availability of the MIL materials. Study 
participants identified positive experiences with m-leaming. They felt that this project gave them 
a new opportunity to learn: about information literacy; an appreciation for the visual aspects of the 
MIL tool; a better understanding of how mobile phones can be used as tools for efficiency; 
accessibility to the vast quantity of information available on the internet (especially Wi-Fi access), 
and the speed with which the lessons could be completed using the tool. 

One common concern voiced by participants regarding the use of smart phones was the cost of 
accessing Internet data. Availability of more Wi-Fi capable phones and accessible locations should 
address the issues of the cost of access. Additional issues raised by the participants with regards to 
MIL training, included: eye strain caused by small mobile screens; difficulty inputting data on 
small keyboard or the phone lacked a keyboard altogether; need to use more multimedia, including 
videos and interactive exercises; MIL web app is only optimized for IOS use, thus causing some 
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viewing issues on Android devices (e.g. Drag and Drop exercises work with touch screen devices); 
and slow Internet connectivity may impact access to videos in MIL lessons. 

Some concerns raised by participants can be addressed by incorporating more sophisticated 
instructional design and by improving the accessibility and functionality of the mobile friendly 
website, while others such as the size of the screen and the limitations on band width are inherent 
limitations of the current technology. Shurtz and von Isenburg (2011) contend that increased screen 
size and touch features on contemporary mobile devices are expected to increase the use of mobile 
technology in education. 

The MIL tool has the potential to be an important aid for m-learning, however to make it more 
effective some enhancements and upgrading of features in the tool and in the lessons will be 
required. Not all MIL contents are fully interactive. Phase 2 of this study/tool will aim to enhance 
student learning and will do further testing of learning analytics. With these enhancements, students 
will be better able to access MIL training resources with interactive module formats on demand 
with multimedia-rich content (such as text, audio, and video),with the aim of increasing learner 
motivation and interest while facilitating more self-directed study. Future implementation of 
HTML5 will allow better animation, smooth video playback, and the capability to run the app and 
store content on the local device even when disconnected from the Internet. We suggest that 
information literacy training could be delivered more effectively if MIL was embedded in the 
curriculum. 

Analysis of the data gathered in the pilot and sharing of the lessons learned from the process will 
benefit future research. Phase 2 will begin in September 2015 with the goal of increasing the 
number and scope of student participants. 


Conclusions 

Notwithstanding the general increase in mobile applications, our research indicates that there is a 
need to collect more information to develop a strong underlying evidence base to support the 
benefits to students of m-leaming and information literacy training. Information literacy is not a 
standard part of classroom content, but appears to be provided only to those students who actively 
seek out the information. Based on our early findings, it appears that students’ information literacy 
knowledge may be linked to specific training. This leads the authors to advocate that information 
literacy be embedded as part of the classroom curriculum. This project has reinforced our belief 
that both learners and educators need to develop a range of information literacy skills and that they 
be provided supportive materials to take full advantage of and make the best use of the emerging 
technologies. 
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